25. Connecting Errors and P-Values

The p-value is the probability of getting our statistic or a more extreme value if the null is true.

Therefore, small p-values suggest our null is not true. Rather, our statistic is likely to have come from a different distribution than the null.

When the p-value is large, we have evidence that our statistic was likely to come from the null hypothesis. Therefore, we do not have evidence to reject the null.

By comparing our p-value to our type I error threshold (\alpha), we can make our decision about which hypothesis we will choose.

pval \leq \alpha \Rightarrow Reject H_0

pval > \alpha \Rightarrow Fail to Reject H_0